Fig. 4a. Error Convergence of the Encoder Networks with Log Error Fig. 4b. Error Convergence for the Encoder Networks with Square Error

We describe the Alopex algorithm as a universal learning algorithm for neural networks. The algorithm is stochastic and it can be used for learning in networks of any topology, including those with feedback. The neurons could contain any transfer function and the learning could involve minimization of any error measure. The efficacy of the algorithm is investigated by applying it on multilayer perceptrons to solve problems such as XOR, parity and encoder. These results are compared with that ones obtained using backpropagation learning algorithm. The scaling properties of Alopex are studied using the encoder problem of different sizes. Taking the specific case of XOR problem, it is shown that a smoother error surface, with fewer local minima could be obtained by using an information theoretic error measure. An appropriate ’annealing’ scheme for the algorithm is described and it is shown that the Alopex can escape out of the local minima.